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Abstract. Let / be the ideal generated by alternating polynomials in two sets of n vari- 
ables. Haiman proved that the q, i-Catalan number is the Hilbert series of the graded vector 
space M{= @ d d2 M^,^) spanned by a minimal set of generators for I. In this paper 
we give simple upper bounds on dim Md lt d 2 m terms of partition numbers, and find all 
bi-degrees (dijdg) such that dimM^ ^ achieve the upper bounds. For such bi-degrees, we 
also find explicit bases for M dl d2 . The main idea is to define and study a nontrivial linear 
map from M to a polynomial ring C[p%, P2, ■ • ■]■ 
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1. Introduction 

In [5], Garsia and Haiman introduced the q, t-Catalan number C n (q, t), and they showed 
that C n (q, 1) agrees with the g-Catalan number defined by Carlitz and Riordan [2]. To be 
more precise, take the n x n square whose southwest corner is (0, 0) and northeast corner is 
(n,n). Let T> n be the collection of Dyck paths, i.e. lattice paths from (0,0) to (n,n) that 
proceed by NORTH or EAST steps and never go below the diagonal. For any Dyck path II, 
let area(Il) be the number of lattice squares below II and strongly above the diagonal. Then 

C n (q,l)= <? area(n) - 
neD„ 

The q, t-Catalan number C n (q,t) also has a combinatorial interpretation using Dyck 
paths. Given a Dyck path II, let a.j(n) be the number of squares in the i-th row that lie 
in the region bounded by II and the diagonal. Garsia and Haglund ([3], [I]) among others 
showed that 

C n (q,t) = g arca{I V inv{n) , 
nez>„ 

where 

dinv(n) := \i < j and a^LI) = a,-(Il)}| + \ i < j and a;(il) + 1 = Oj(n)}|. 

A very natural question is to find the coefficient of q dl t d2 in C n (q, t) for each pair (d\, d 2 ), 
in other words, to count how many Dyck paths have the same statistics (area, dinv). It is 
well-known that the sum area(II) + dinv(II) is at most (™) . In this paper we find coefficients 
of q dl t d2 when (™) — d\ — d 2 is relatively small. 

Denote by p(k) the partition number of k and by convention p(0) = 1 and p(k) = 
for k < 0. Denote by p(b, k) the partition number of k into no more than b parts, and by 
convention p(0, k) = for k > 0, p(b, 0) = 1 for b > 0. One of our main results is as follows. 

Theorem A. Let di,d 2 be non-negative integers di,d 2 with d\ + d 2 < (2)- Define k = 
(2) — d\ — d 2 and 5 = min(c?i, d 2 ). Then the coefficient of q dl t d2 in C n (q,t) is less than or 
equal to p(8, k), and the equality holds if and only if one the following conditions holds: 

• k < n — 3, or 

• k = n — 2 and 5 = 1, or 

• 5 = 0. 

This theorem is a consequence of Theorem C. It contains [10, Theorem 6] and a result of 
Bergeron and Chen [U Corollary 8.3.1] as special cases. In fact it proves [TOl Conjecture 8]. 
We feel that the coefficient of q dl t d2 for general k can also be expressed in terms of partition 
numbers, only that the expression might be complicated. For example, we give the following 
conjecture which is verified for 6 < n < 10. 
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Conjecture. Let n, d\, d,2, 5, k be as in Theorem |A] If n — 2 < k < 2n — 8 and 5 > k, then 
the coefficient of q dl t d2 equals 

p(k) - 2[p(0) + p(l) H hp(A; - n + 1)] - p(k - n + 2). 



As a corollary of Theorem A, we can compute some higher degree terms of the special- 
ization at t — q. 

Corollary B. 

C n (q, q) = ^ ^p(fc) ~ 3 ^ + ^ + 2 q^~ k + (lower degree terms). 

From the perspective of commutative algebra, the q, t-Catalan number is closely related 
to the graded ideal I defining the diagonal locus of (C 2 )™. In [5] and [TJ, Haiman proved that 
the q, i-Catalan number is the Hilbert series of the graded vector space spanned by minimal 
generators for /. Blowing up the ideal / gives the well-known isospectral Hilbert scheme 
discovered by Haiman in his proof of the n! conjecture and the positivity conjecture for the 
Kostka-Macdonald coefficients [6]. A natural question, posed by Haiman [8], is to study a 
minimal set of generators of the ideal /. An extensive study of generators of I might lead 
to an explicit principalization of the ideal I. 

To construct a minimal set of generators of I is difficult. However, if we focus on cases 
when the degree is Q) — k where k < n — 3, we can give an explicit combinatorial description 
for a minimal set of generators. 

Now we turn to a detailed description. Fix a positive integer n. Consider n-tuples of 
ordered points Ui)}i<i< n in the plane C 2 . The set of all n-tuples forms an affine space 
(C 2 ) n with coordinate ring C[x, y] = C[xi, yi, ...,x n , y n ]. Denote by C[x, y] £ the vector space 
of alternating polynomials spanned by a basis {A(D)} _d g d„ defined as follows. Denote by N 
the set of nonnegative integers. Let D n be the set of subsets D = {(ai,/3i), (a n , /3 n )} of 
NxN. For D G D„, define 



A(D):= det 



x n yn ~°n tin "• x n tin 



The ideal I C C[x, y] is the radical ideal that defines the locus where at least two points 
coincide, to be precise, 

l<i<j<n 

Haiman [6] has proved that / is in fact generated by C[x, y] e , therefore is generated by 
{MP)\n:v,, 

Finding a minimal set of generators of / is equivalent to finding a basis of M := J/(x, y)/ 
where (x, y) is the maximal ideal (xi,yi, . . . ,x n ,y n ). Since M is naturally bi-graded with 
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respect to x-degree and y-degree, we can write 

di,d,2 

In [3 p393], Haiman discovered the amazing fact that 

(1-1) C n (q,t) = ^tV 2 dimM dlid2 . 

d\,dl 

Setting q — t — 1, we get 

,• i f 2 A 

dim c M = — — 

n + 1 \ n J 

which is the usual Catalan number C n . 

The authors showed in [10] that, when the deficit k = (™) — d\ — d 2 is relatively small 
compared to n and di,d 2 are not too small, an explicit basis of M dljd2 can be constructed 
in one-to-one correspondence with partitions of k, by using what we call minimal staircase 
forms. However the bound of k given in [10] was by no means sharp. In this paper we find 
all bi-degrees (di,d 2 ) for which dim M dl>d2 are exactly partition numbers of k into no more 
than mm(di,d2) parts. For such bi-degrees, we also find bases for M dl)d2 . 

Theorem C. Let di,d 2 be non-negative integers d\ 1 d 2 with di + d 2 < (2)- Define k = 
(2) — di — d 2 and S = min(di, d 2 ). Then dimM dljd2 < p(5, k), and the equality holds if and 
only if one the following conditions holds: 

• k < n — 3, or 

• k = n — 2 and 5 = 1, or 

• 5 = 0. 

In case the equality holds, there is an explicit construction of a basis of M dlid2 . 

The theorem is a consequence of Theorem [2H and Theorem [33 Obviously, Theorem Rl 
immediately follows from Theorem [UJ thanks to (II. ip . a theorem of Haiman. The idea of 
the construction consists of two parts: the easier part is to show 

dimM dl42 < p{S,k) 

using a new characterization of q, t-Catalan numbers; the harder part is to construct a set of 
p(S, k) linearly independent elements in M dltd2 . It seems difficult (as least to the authors) to 
test directly whether a given set of elements in M dl d2 are linearly independent. We define a 
map ip sending an alternating polynomial / G C[x, y] e to a polynomial ring C[pi, p 2 , p%, . . .]. 
The map has two desirable properties: (i) for many /, <p(f) can be easily computed, and (ii) 
for each bi-degree (di,d 2 ), <p induces a morphism (p : M dlt d 2 — > C[p\,p 2 , ...} of C-modules. 
Then we use the fact the linear dependency is easier to check in C[pi, p 2 , ...} than in M dlt d 2 - 
This idea is motivated by our earlier work [TO] . 

The structure of the paper is as follows. After introducing some notations in §2, we 
define and study the map (p in §3, then in §4 and §5 we give the upper bound and the lower 
bound of dimM dljd2 , and prove the main result in §6. For readers' convenience, we give 
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the table of q, t-Catalan numbers for n = 7 in appendix §7.1 and a Macaulay 2 code for 
computing the map ip in §7.2. 

Acknowledgements. We are grateful to Frangois Bergeron, Mahir Can, Jim Haglund, Nick 
Loehr, Alex Woo and Alex Yong for valuable discussions and correspondence. 

2. Notation 

• We adopt the convention that N is the set of natural numbers including zero, and 
N + is the set of positive integers. 

• For n G N + , define D n = {D = {(ai, bi), . . . , (a n , b n )}\di, bi G N}, i.e. an element of 
D n is an ordered set D of n points in N x N. Define £> = W^ =1 D n . Similarly, define 
SI = {D = {(ai, 6i), . . . , (a n , b n )}\ ai G Z, b t G N, a, + b { > 0}. Define £' = U™ =1 & n . 

We use Pi to denote the point (a^, bi), and denote \Pi\ = cii+bi, \Pi\ x = \Pi\ y = b; L . 
Unless otherwise specified, we assume throughout the paper that 

(2.1) P 1 <P 2 <---<P n , ioiD = {P 1 ,...,P n } 

where the order is defined as follows: 

(a, b) < (a', b') if a + b < a' + b' , or if a + b = a' + b' and a < a'. 

In particular, \Pi\ < \P 2 \ < • • • < \P n \. 

• Given a monomial / = x^y^ 1 ■ ■ ■ x^ n y^ n G C[x,y], we call (^"=1 a ^ Ya=i A) tlie 
bi-degree of /. A polynomial in C[x, y] is bi-homogeneous of bi-degree (d 1: d 2 ) if all 
its monomials have the same bi-degree (di,d 2 ). 

Given D = {(ai, . . . , (a„, b n )} G D„, we call (XT=i a i' Z)"=i tne bi-degree of 
D, which is the same as the bi-degree of the polynomial A(D). 

• Let k, b G N + . Denote the set of partitions of k as 

n fc = {v = (vi, v 2 , •••)\v i e N + , v x < v 2 < ■ ■ ■ , and v x + v 2 -\ = k)}. 

Denote by 11^, ^ the set of partitions of k into at most b parts. 

Define the partition numbers p(k) = #n fe and p(b,k) = #n 6;fe . By convention 
p(0) = 0, p(0, fc) = for k > 0, p(b, 0) = 1 for all b > 0. 

• Let Z[p] = Z[pi,p 2 ,---] be the polynomial ring with countably many variables 

pi, p 2 , By convention we assume po — 1. For a partition z/ = (z/i, i/ 2 , • • • ) G 11^, 

define p v = p Ul p U2 • • • G Z[p]. 

• For n G N + , denote by S n the permutation group of {1, ...,n}. 

• Given two bi-homogeneous polynomial /, g of bi-degree (di,d 2 ), let /, g be the cor- 
responding element in Md lt d 2 - We say that / = g (modulo lower degrees) if / = g in 
M dl , d2 . 

3. Map (p. 

3.1. Definition and properties of <p. In this subsection we define and study the map 
(p which naturally arises when we look for a minimal set of generators of the ideal / of 
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alternating polynomials. For readers' convenience, a Macaulay 2 code for computing <p is 
put in Appendix. 

Definition 1. (a) Define the map <p : D' n — > Z[p] as follows. Let D = {(ai, &i), (a„, 6 n )} G 
= © " Er=i(«i + *>i), and define 

ip(D) := (-l) fc ^ sgn(cr) JJ (^p Wl p W2 • • -p^.) , 

creS n 1 = 1 

where (wi, . . . , tu&J in the sum p Wl p W2 • • ' P«v runs through the set 
(3.1) {(u>i,...,w fe J G N b <| wi + ... + = <r(i) - 1-Qi-bi}, 

with the convention that 

{0 if a{i) - 1 - Oj - 6j < 0; 
if 6j = and cr(z) - 1 - a { - b { > 0; 
1 if bi — and cr(z) — 1 — a, — 6j = 0. 

(b) Here is an equivalent definition of tp(D). Define the weight of p« to be % for i G N + 
and define the weight of po = 1 to be 0. Naturally the weight of any monomial cpi t ...pi n 
(c G Z) is defined to be %\ + ... + i n . For w G N and a power series / G Z[[p 1; p 2 , •••]], denote 
by {/} w the sum of terms of weight-w in /, which is a polynomial. Define 





h(b, w) : = 


{(1+P1+P2 + 


■••) 6 } w! 


beN,w 


G Z. 




Naturally h(b, w) 


= if w < 0. 


Also assume (1 


+ Pl+P2 + ---)° = 


; 1. Then 




p(D) = (-l) k 


h(b u - 
h(b 2 ,- 


Pi\) 
A|) 




Pi\) 
A|) 


h(b u 2- 
h(b 2 ,2- 


Pl\) ■■■ 

Pal) ••• 


h(bi,n — 1 — 
h(b 2 ,n- 1 - 


AI) 




h(b n , - 


Pn\) 


h(b n , 1 - 


Pn\) 


h(b n ,2- 


Pn\) ■ ■ 


h(b n ,n- 1 - 


A|) 



(c) Let Di, . . . , Dz G D' be of the same bi-degree and ^=1 C «A ^e the formal sum for 
any q G C (1 < % < t). Define 

l £ 

p(^2 °i D i) := ^2 Ci ^(A)- 
i=l i=l 

For any bi-homogeneous alternating polynomials / = Ym=i c i A(_D;) £ C[x, y] e , we define 

1 1 
¥>(/) ■= vC^CiDi) = ^Q<p(A) 

i=l i=l 

by abuse of notation. □ 



Before relating <p(D) with A(D), we shall first look at some properties of the map ip. 
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Lemma 2. Let n G N + ; D = {Pi, . . . , P n } G D' n where P\ < ... < P n as in the assumption 

a. 



(i) If \Pi\ > i for some 1 < i < n, then ip(D) = 0. 

(ii) Let m G N + and Q\, . . . , Q m GZxN satisfy \Qi\ = i — 1 for 1 < i < m. Let 

D = {Qi, ...,Q m ,Pi+ (m, 0), P 2 + (m, 0), . . . , P n + (m, 0)}. 
Then <p(D) = <p(D). 

(iii) Lett G N+, Q = (-t,t) and /) = {Pi + Q, P 2 + Q, . . .,P n + Q}. Then 

<p(D) = ip(D). 

(iv) Let S — {i | |Pj| := ai + h = i — 1} = {ii < ■ ■ ■ < i^ and assume i\ — 1. VFe define 
the set {Pi r , . . . , Pj r+1 -i} the r-th block o/ P /or 1 < r < £ (assuming Pi e+1 = n + 1). Then 

t 

<p(D) = J] ip{{P ir - (i r - 1, 0), P lr+1 - (v - 1,0),..., P Wl _! - (v - 1,0)}). 

r=l 

(v) Suppose \Pi\ = /or 1 < i < n. Then f(D) = c ■ p\ 2 for a positive integer c. In 



fact, 



l!2!---(n- 1)!' 

(vi) For s G N+, let D = {(-1, 1), (0, 0), (1, 0), . . . , (s - 1, 0))}. Then 

(pip) = ps- 

Before giving the proof, let us look at some examples explaining the lemma. 
Example 3. (i) We have <p({(0, 0), (1, 0), (2, 1), (3, 0)}) = since |P 3 | = 2 + 1 = 3. 
(ii) Let D = {(-1, 1), (0, 0), (0, 1)}, m = 2, Q 1 = (0, 0), Q 2 = (1, 0). Then 
^({(0, 0), (1, 0), (1, 1), (2, 0), (2, 1)}) = <p{D), 



i.e. 



■)■ 



(iii) Let D = {(0, 0), (0, 1), (1, 0)}, t = 1. Then 



■)=^(- 



■)• 



(iv) Let P = {P 1; P 2 , P 3 , P 4 , P 5 , P 6 } = {(0, 0), (0, 1), (1, 0), (2, 1), (3, 0), (2, 2)}. There are 
3 blocks in P, namely {Pi}, {P2,Ps} and {P 4 ,P 5 ,P 6 }. Then 

<p(D) = ^({(0, 0)}) • p({(-l, 1), (0, 0)}) • ^({(-1, 1), (0, 0), (-1, 2)}), 



i.e. 
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■)•¥>(■ 



■)• 



(v) For D = {(— n + 1, n — 1), (— n + 2, n — 2), . . . , (-1, 1), (0, 0)}, p(£>) = pf 

Proof of Lemma\^ (i) It immediately follows from the condition fl3.ll) . 
(ii) By definition, 



<p(D) 

where Wi, . . . , G N and 

Wi + ...+W b , 



E 



m+ra 

sgn(a) JJ ^ p Wl • • • p 
i=l 



/ ' 



ai - 6,- 



|Pi. 



if z < m; 
if z > m. 



If er(z) < z for some i < m, then nowj,..., u>&. satisfies the condition, ni=i n (X] Pm ' ' ' P 
0, hence the summand corresponding to a does not contribute to (p(D). So we only need to 
consider those a satisfying a(i) = i (1 < z < m). Each such a corresponds to a permutation 
of {m + 1, . . . , m + n}, and by translation, a permutation of {1, . . . , n}. To be precise, 



(71 Z — 771 



O Z 



m, m + l<i<m + n. 



Then a(i) — 1 — m — IP,-. 



a(i — m) — 1 — I Pf. 



for m + l<z<m + n. Moreover, 

71 



"a m )-Ew.i-E<ifli+m) = (a)-Eifli = * 

/ i=i i=i \ ' i=i 

Comparing with the definition of ip(D), we conclude that <p(-D) = (p(D). 
(iii) It suffices to prove the case when £ = 1. Define 



~h(bi,i — 


PiDl 




"/i(6i + 


PI)] 


h(b 2l i - 


p 2 |) 




/i(6a + l,i- 


p 2 |) 


h(b n ,i- 


Pnl). 




_/i(6 n + l,z - 


Pnl). 



< i < n- 1. 



By the definition of the map tp, 

<p(D) = (-l) k det(v , . . . , v n _i), <p(D) = (-l) fc detK, . . . , 
By the definition of the function h, it is easy to deduce the relation 

h(b + 1, w) = h(b, w) + pxh{b, w - 1) + p 2 h(b, w - 2) H . 

. , \P n \ are non-negative integers, the above relation implies 
w[ = Vi + piVi-i + p 2 Vi_ 2 H h PiV , < i < n - 1, 



Since |Pi 
hence 



•l) fc det(v , 



v„_i 



•l) fc detK,...,vU)=Vp)- 
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(iv) Suppose the summand in (p(D) corresponding to a e S n does contribute. By the 
definition of tp(D), it is necessary that a(j) — 1 — \Pj\ > for 1 < j < n. Let 1 < r < t. For 
j > i n we have |P,-| > |Pi r |, therefore 

<70')>i + l^-|>i + l^ r | = v 

So (j maps the set {i r: i r + 1, . . . , n} to itself for every r. It follows that a maps each block 
to itself. Let ay be the restriction of <r to {i r ,i r + 1, . . . ,i r +i — !}■ Define n r = i r+ i — i r , 
k r = Y^j=l~-\3 ~~ Y^j=l~ Then by (ii) and a routine computation, we have 

niH hnj> 



= (-l)^ + - +fc ' £ 8gn( ffl )... 8 gn(^) (E^-^w) 

(Tl,...,<T^ i=l 

£ , n r x 

= n ( (-i^E^^n (e^-^w) ) 

r=l ^ ay i=l ' 

= II ^({ P V - -1,0), P r+ l - (i r - 1, 0), ... , P ir+1 _! - (i r - 1, 0)}). 



r=l 



(v) We rewrite the definition of tp as 



(3-2) V(D) = (-I) k E S ^)I]^^-^ ' 



(<T,{l»f }) V 

where is a set of nonnegative integers, 1 < i < n, 1 < j < &j. For 1 < « < n, since 

|P | =0, those satisfy the condition 

+ ■ ■ ■ + = a(i)-l. 

Denote by £ the set of all possible data (<7, Let £' C S be the subset consisting of 

those (a, such that not all are or 1. We shall define a 'conjugation' on the set 

£', i.e. an automorphism /:£'—> E' such that / o / is the identity. 

For (<t, {w^}) G S', define to be the number of nonzero elements in (tuj, . . . , w^), 
for 1 < % < n. Then 



mi H h m n < + 1 H h (n - 1) 



Since some u>j^ is greater than 1, the inequality must be strict, therefore we can find a 
smallest pair (r, r') such that r < r' and m r = m r r. (Here we use the lexicographic order, 
i.e., (r, r') < (s, s') if r < s or (r = s and r' < s').) Let 

{ji<---<j mr }:= {j\w\ r) ^0}, 
{ji<-<jU^{j>fV0}. 
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as follows. For i ^ r, r', define wf 1 = wf for 1 < j < hi. For % = r, define 



w[ r ? = w[ r f ^ for 1 < i < m r , 



Define a = a ■ (r, r'), i.e. cr(r) = a(r'),a(r') = a(r), a(£) = <j(£) for I ^ r,r'. Define {w^} 

,. ' for 1 <j < hi. For i = r, define 

for j ji, . . . ,j mr . 

Similarly for i = r', define 

o' t ] = w h for 1 - £ - m " and } = for j' 7^ j'i , . . . , j' mr . 

Define the conjugation / : (cr, {w^}) i— > (cr,{-u;^}). It is immediate from the above con- 
struction that / is a conjugation. Moreover, / has no fixed point because a ^ a. Since 
sgn(a) = — sgn(cr), the summand in (13. 2p corresponding to (a, {w^}) cancels with the sum- 
mand corresponding to (a, {w^}). 

Finally, we are left with the case when all are or 1. Using Definition [T] (b), and 
using the fact that the monomial p™ in h(b,w) has coefficient ( w ), we obtain 

ftw oi ■■■ e>r ft) ft) ■■• C-x) 

ft) ft) ■•■ ej 



<^(L>) 



Pi 



c-pi 



ftM ftM ■■• (ft>r ] 

where c is the second determinant. Notice that Q = 6(6 — 1) • ■ • (6 — ? + is a polynomial 
of 6 of degree i whose leading term is b l /i\. By appropriate column operations, i.e., adding 
appropriate multiples of the first i — 1 columns to the i-th column for 1 < i < n, we obtain 

I n 
On 



ft) ft) 
ft) ft) 



C-l) 
(ftJ 



1 6 & 

x "n 2! 



1 6i I 



(n-l)! 



1!2!--- n-11! 



2! (n-l)! 

by using standard results of Vandermonde matrices. Since 6 X > 6 2 > ■ • • > b n are distinct 
integers by assumption, c is a strictly positive integer. 

(vi) Follows immediately from Definition [1] (b). □ 

3.2. Relation between tp(D) and A(D). We need the following elementary lemma. 
Lemma 4 ([ID]. Lemma 26). For GN x N (1 <i <nj and c, e e N, 



i=l 



x i £/i 


x?y? 2 ■ 

X2V2 2 ■ 




n 

= £ 

i=i 


x 2 2/2 


x l i/l 

x 2 i/2 




tin 






x n i/n 


ai+c Pi+e _ 





y4s a consequence, 

n 

= A({(ai, A), . . . , (oi_i, A-i), («i + c, /3j + e), (a i+1 , /? <+1 ), . . . , (a n , /?„)}) 



i=l 
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modulo lower degrees. 



Let us recall the definition of minimal staircase forms defined in [10], and then define 
special minimal staircase forms. 

Definition 5. We call D = {P 1; . . . , P n } G D n a minimal staircase form if |Pj| = i — 1 or 
2 — 2 for every 1 < i < n. For a minimal staircase form D, let {ii < z 2 < ■ ■ • < h} be 
the set of i's such that |Pj| = i — 1, we define the partition type of P to be the partition of 
((2) — S l-^il ) consisting of all the positive integers in the sequence 



(ii - l,z 2 - ii - l,i 3 -*2 - 1, • 



^ - ^ 



_i - l,n- ^) 



Example 6. Forn = 8 and P = {Pi, . . . ,P 8 } satisfying (|Pi|, . . . , |P 8 |) = (0,1,1,2,4,4,5,6), 
the set {i |p| = i — 1} equals {1, 2, 5}. The positive integers in the sequence (1 — 1,2 — 1 — 
1,5 — 2 — 1,8 — 5) is (2, 3), so the partition type of P is (2, 3). 

Definition 7. The data (m, n, (r x , . . . , r m ), (si, . . . , s m )) GNx (N + ) x N m x N ra satisfying 
1 < Ti < r 2 < • • • < r m < r m+ i := n and < Sj < r i+ i — ?-j — 1 (1 < z < m) determines a 
D G S) n as follows. 

D = {(0, 0), (1, 0), • ■ ■ , (n - 1, 0)} U {(n - 1, 1), (r 2 - 1, 1), ... , (r m - 1, 1)} 

\ {(n + si, 0), (r 2 + s 2 , 0), . . . , (r m + s m , 0)}. 

We call P a special minimal staircase form. 

Remark 8. It is easy to see that a special minimal staircase form is indeed a minimal 
staircase form. Using the notation in the definition, the partition type of a special minimal 
staircase form D is obtained from (si, s 2 , . . . , s m ) by eliminating 0's and sorting the sequence 
if necessary. The following picture gives a typical example of a special minimal staircase form, 



• • • • • 



where m — 3, n = 13, (r 1; r 2 , r 3 ) = (2, 5, 7), (si, s 2 , s 3 ) = (2, 1, 5), and the partition type is 
(1,2,5). 



Let us recall the following two facts proved in [TU] . 

Lemma 9 (Minors Permuting Lemma in [10]). Let D = {Pi,...,P n } G D , h,£,m G N + 
satisfy 2 <h< h+£+m < n + 1, \Ph\ = h—1, \Ph+e\ = h+£—l, \Ph+e+ m \ = h+£+m — l (this 
condition holds ifh + £ + m = n + l by assumption) . Suppose \P h+ e\ x , \Ph+e+ m -i\ x > t- 
Define 

d' = {p x , p 2 , . . . , iVi, - 0), p^+i - 0), . . . , p ft+ £ +m -i - {e, 0), 

P ft + (m, 0), P/,+1 + (m, 0), . . . , Pft+^-i + (m, 0), P^+^+m, . . . , P n }. 
T/ien A(P) = A(P') (modulo lower degrees). 

Lemma 10 (Main Theorem of [10J ) . Suppose k is a positive integer such that n > 8k + 5 and 
d±, di > (2k + l)n are two integers whose sum is n(n — l)/2 — k. Then M dl ^ 2 is minimally 
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generated by p(k) elements, i.e., dim = p(k). Furthermore, there is a one-to-one 
correspondence between partitions of k and generators, namely 

(/i = '^^niiji G life) < > (a minimal staircase form of partition type ft). 

The following lemma is essential for this paper. 

Lemma 11. (i) Let d\, d 2 ,kEN and d\ + d 2 = (2) — k. Define 

, _ ( there exists a minimal staircase form G D n of 1 

k 1 ^ k partition type \i and of bi-degree (di,d 2 ) J ' 

If there are coefficients a^eC (ft G H' k ) satisfying 



/ a fl A(F fl ) = (modulo lower degrees), 
Men; 

t/ien = 0, V/x G H' k . In other words, {A(F /i )} /ien ' form a linearly independent set in 

(ii) Any two special minimal staircase form in D n of the same partition type and the 
same bi-degree are equivalent modulo lower degrees. 

Proof, (i) Choose a sufficiently large integer N and choose (N — n) points P n +i, • • • , Pn £ 
NxN such that \Pi\ = i - 1 for n + 1 < % < N and 

\P n +i\x + ■■■ + \Pn\x > (2k + 1)N, \P n+1 \ y + ■■■ + \P N \ y > (2k + l)N. 

let F'^ = F M U {P n+ i, P n +2, ■ ■ ■ , Pn}- The condition of Lemma [TD] is satisfied, so A(F') for 
/i G tl' k are linearly independent modulo lower degrees. But A(F') is equivalent to A(F M ) ■ f 
for a polynomial /o independent of \i. (In fact /o = Yli j a ij for 1 < i < j < N and j > n+ 1, 
with appropriate choices a^- = — or — We do not need the exact formula here.) 
So the linear independence of {A(P^)} M6n -' implies the linear independence of {A(F^)} fien > k - 

(ii) The claim follows immediately from Minors Permuting Lemma (Lemma [9]). □ 

Proposition 12. Let n G N+ ; D = {P 1; . . . , P„} e ® and k = g) - £]" =1 |P;| > 0. Suppose 
N G N+ safe/^es N > N := {J2i =1 \Pi\ y )(k + I) . Define 

D = {(0, 0), (1, 0), . . . , (iV - 1, 0), P x + (iV, 0), . . . , P n + (N, 0)} G IW 

Lei c?2 = |Pj|?/ &e the y-degree of D (which is also the y-degree of D). For /j G ILj 2) fc ; /ei 
Fp be a special minimal staircase form with the same bi-degree as D and be of partition type 
fi. Then there exist unique integers (\l G ILj 2j aJ such that 

A(D) = ■ A(F fJi ) (modulo lower degrees). 

In fact, the integers a M satisfy 

(3.3) (f(D) = a ^/V 

MGn d2ifc 
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Proof. In this proof we use D e D that does not satisfy the assumption (12.11) . 

The uniqueness of a M follows from the fact that {A(F A1 )} Men / fc form a linearly independent 
set in M^da, proved in Lemma [TT1 For the existence of a^, we shall give an algorithm showing 
that those a^i sire exactly the integers satisfying (13.31) . 

We separate the set {(1, 0), (2, 0), ... , (N , 0)} into (J^Li \Pi\y) segments, where the r-th 
segment for 1 < r < (J^Li \Pi\y) consists of {k + 1) points {(i, 0)| (r — l)(k + 1) + 1 < i < 
/•(/•• • l)j. 

Consider the following sequence of length <i 2 . 

(*) : (1, |P4), . . . , (1, 2), (1, 1), (2, \P 2 \ y ) , . . . , (2, 2), (2, 1), . . . , (n, \P n \ y ), . . . , (n, 2), (n, 1) 

with the natural total order that (i', f) < if (z', f) is to the left of (i, j). For in 
the above sequence, define r(i,j) 6 N + to be the integer such that(z, j) is the r(z, j)-th pair 
in the sequence (*). 

Denote Qi 0) = (s - 1, 0) for 1 < s < iV and P 4 (0) = P + (JV, 0) for 1 < t < n and denote 

:=£>= {Q®, Qf\..., Q®, Pl°\ Pf, . . . , Pf}- 

Given a set of nonnegative integers w = {w^, ^}(i'j')eO)> we construct 

D^ = {Q^...,QP,Pi r \...,P^} 

inductively on r e [l>X/"=i l-^U- Here we do not require P/ r ) to satisfy the assumption 
of order defined in (12. ip . Suppose p( r_1 ) has been constructed and the r-th element in the 
sequence (*) is the pair (i, j). Then P^) is constructed as follows. 

Q (r) m = ((r-l)(Jfe + l),l); 

Q( p) = Q< r_1) , for 1 < £ < N and £ ^ (r - l)(k + 1) + 2 + wf\ 

i 5 l w = ^" 1) + M !) + i,-i); 

= p/ r_1) , for l<£<nand£^i. 

The following can be proved inductively on r: 
(3.4) A(P>) = (-l) r A C°w )> (modulo lower degrees) 

w 

where w runs through all possible sets of integers {w^, ^}(i',j')<(i,j) where w^, ^ G [0, k]. Indeed, 
for r = 1 and \Pi\ y > (the case \P\\ y = is similar) we need to show that (for simplicity 
of notation we use w in place of ) 

A(D)+ A({(0, 0), . . . , (w, 0), (0, 1), (w+2, 0), . . . , (N-l, 0), P^w+1, -1), P 2 , . . . , P n }) 

0<«)<fc 

is equivalent to modulo lower degrees. But this is an immediate consequence of Lemma H] 
by plugging in (c, e) = P^ — (0, 1) and 

{fa, A), (a 2 , ft), . . . } = {(0, 0), (1, 0), . . . , (N - 1, 0), (0, 1), P 2 (0) , P 3 (0) , . . . , Pf}. 
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Here we can assume w < k because otherwise the total degree of the polynomial A(D) is 
strictly greater than ( Ar ^" n ) hence A(D) = modulo lower degrees. 

For r = 2, we only consider the case \Pi\ y > 2, since the other case is similar. By 
induction we have 

A(D) = — AfD^ls ), (modulo lower degrees). 

ni i 1 lit/ 

n ly 

By a similar argument as in the case r = 1, 

A(D 'I ) = — A(D ( H fll ), (modulo lower degrees). 

w ip, i W p, i ' w ip, i 1 1 

nly fii , I -n.il/ nly- 1 

I -n. I » — j. 

Combine the above two formulas together, we have 

A(P) = (-1) 2 V A(P (1) (1) (1) ), (modulo lower degrees). 

ID (i) Inly nly- 1J 

— Inly Inly- 1- 

An easy induction similar to the above argument gives the proof of (13.41) . 



Now look at (13.41) when r = r = 

ELi The ^/-coordinates of P^, . . . , P (ro) 



, J- n 



arc 



all zero. A necessary condition for A(D { Z o) ) £ is that 

{\Pi ro) \ x , I P 2 (ro) U, • • • , |^ ro) U} is a permutation of {iV, N +1,...,N + n— 1} 

and hence we can assume such a condition holds. Let a 6 5* n be the permutation that 
satisfies ||P/ ro) U = <r(i) + JV - 1. Since 

i=i 

we have 

|n.|-y 

wf = Pf o) - P/ 0) = (<r(i) + iV - 1) - (N + |P|) = - 1 - \Pi\ = a(i) - 1 - a, - b h 

3=1 

which is exactly the condition in the definition of ip(D) (cf. Definition [I](a)). Next, we shall 
figure out the correct sign. For this, we have to rearrange the order of points in Dw ' to 
satisfy the condition (12. ip . For 1 < r < Yle=i I-^Hvj the r "^ n segment 

((r-l)(A; + l) + l,0),((r-l)(A; + l)+2,0),...,((r-l)(A; + l) + l+u;f,0),...,(r(A; + l),0) 
is modified to 

(( r _ i)( fc + i) + i ; o), ((r - l)(k + 1) + 2, 0), . . . , ((r - l)(k + 1), 1), . . . , (r(k + 1), 0). 

The only change is that the point [(r — l)(k + l) + l + w^\o) is replaced by ((r — l)(/c + l), l). 
To rearrange this segment into correct order, we need to move the (1 + Wj)-ih point in 
front of the first point, so the change of sign is {—l) w j . On the other hand, rearranging 



DIAGONAL IDEALS 



15 



{Pi 
is 



Pi} to the correct order incurs a sign change sgn(a). So the overall sign change 



gn^aj 



which coincides with the signs in the definition of <p(D) (cf. Definition QJa)). 

Finally, note that Dw ' (after rearranging it to the correct order) is a special minimal 
staircase form defined in Definition [7J The partition type of Dw ' is (Wj)ij, which is 
compatible with the definition (13. 2p of ip(D). Thus we have finished the proof of Proposition 

□ 



4. The upper bound of dimM dltd2 

4.1. A characterization of the q, t-Catalan number. Recall the following conjecture we 
gave in [TO] . We would like to point out that Mahir Can and Nick Loehr gave an equivalent 
conjecture in their unpublished work. 

Conjecture 13. Let A n be the set of integer sequences Ai > • • • > A n _i > A n = satisfying 
Xi < n — i for all i G [1, n]. For any A = (Ai, . . . , A n ) G A n , let 

at = n - i - A i; k = #{j| i < j < n, Xi - Xj + i - j e {0, 1}} 

and define D(X) = {(aj,fcj)|l <i< n}. Then {A(D(X))}\eA n generates the ideal I. 

Example 14. For n = 3, A 3 consists of (2, 1, 0), (1, 1, 0), (2, 0, 0), (1, 0, 0), (0, 0, 0), the corre- 
sponding D(X) are 



*— » 



We shall not prove the conjecture in this paper. Instead, we give a characterization of 
D(X) appeared in the conjecture. This characterization will be used to provide an upper 
bound of dimM dl C z 2 . 

Definition 15. Let I j)™ talan be the set consisting of D C N x N, where D contains n points 
satisfying the following conditions. 

(a) If (p,0) G D then (i, 0) G D, Vi G [0,p]. 

(b) For any pGN, 

#{j I (P + 1, J) e ^} + #{J I (P,J) eD}> max{j | (p, j) ED} + 1. 
(If {j | (p,j) G = 0, then we require that no point (i, j) G D satisfies i > p.) 



Proposition 16. T7ie map 6 1 : A n — > r £) c ^ talan sending X to D(X) is one-to-one. 

Proof. We first show that D(X) is in r £) c ^ talan ( i. e ., it satisfies conditions (a)(b) of Definition 
[13 
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By the definition of D(X), suppose a, = for some G [l,n], then i < i' if and only 
if 6j > &.;/. Indeed, suppose z < i! . Since a« = implies (Aj + z) — (Aj/ + z'), we have 

{j\i<j< n, (Aj + - (Aj + j) G {0, 1}} D {j\ i'<j< n, (Aj/ + z') - (A, + j) G {0, 1}}, 

hence fej > 

For (a), suppose (ag, 6^) = (p, 0) 6 -D(A) and (p — 1, 0) ^ -D(A). Since 

a,i - a i+1 = (n - i - A») - (n - % - 1 - Aj + i) = 1 - (Aj - A i+ i) < 1, Vz' G [1, n - 1] 

and a n = 0, there exists z G + 1, n] such that aj = p — 1. Suppose zo is maximal among all 
such z". Since (a^, = (p, 0), we have a, < p for all z > £ Therefore 

= #01 io<3< n, aj - a io G {0, 1}} = i < j < n, % G {p - l,p}} = 0, 
(a io , 6 io ) = (p — 1, 0), which contradicts our assumption that (p — 1,0)^ -D(A). 

For (b), if {j | (p, j) G £>} = 0, then since aj — dj + i < 1 Vz, there is no point in D 
whose x-coordinate is greater than or equal to p. Now assume {j \ (p,j) G D} ^ 0, define 
q = max{j | (p, j) G D}, and (a^, 6^) = (p, q) G £>. By the definition of b^, 

Q = h = #{j\£ <3 < n , a-j ~a e e {0, 1}} = < j <n, aj =p or p+ 1}, 

therefore, 

#{j I (p + 1, j) eD} + #{j | (p, j) eD}>q + l. 
So -D(A) is in T)™ talan . 

To show that 9 : £) i— ► -D(A) is a bijection, it suffices to construct a map sending 
D(X) back to A. We give an inductive construction on n. Let p G N be the minimal integer 
such that 

#{j I (p + 1, J) G ^} + #{j | (P, j) e D} < max{j | (p, j) E D} + 1. 

Let g = max{j | (p, j) G .D} and define (ai,b\) = (p,q) G D. Now D' := D \ {(ai,6i)} 
has (n — 1) points and we can check that it is in 

^Catalan _ By i nc i U ction we have 6~ 1 (D') = 

(X[, X' 2 , ■ ■ • , Ajj_ x ). Then we define 

r 1 (D) = (n-l-p,A' 1 ,A / 2 ,...,A^_ 1 ). 

To check that it is in A n , we need to show n — 1 — p > X[, i.e., p < (n — 1) — X[ — a[ + 1, 
where a'± is the minimal integer that 

#{j | + 1, j) G D'} + #{j | (a[,j) G £)'} < max{j | (a[,j) G £)'} + 1. 

But D and D' coincide on column 0, 1, ... ,p — 1, therefore a[ > p — 1. 

To check that 9 and 6~ l are inverse to each other is routine and we shall skip. □ 

Remark 17. The above proposition is also discovered independently by Alexander Woo 

Corollary 18. The dimension of M^^, i.e., the coefficient of q dl t d2 in the q,t- Catalan 
number C n (q,t), is equal to the number of D G f £> c ^ talan such that the x-degree (resp. y- 
degree) of D is d\ (resp. d 2 ). 
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Proof. It is an immediate consequence of Proposition [T6], by using Garsia and Haglund's 
description of q, t-Catalan number ([3], [1]), which asserts, in notations of Conjecture [T3| 
that 

C n (q,t) = £ g S>tS> . 
aga 

□ 



4.2. The upper bound of dim M dltd2 . hi order to compare M dljd2 f° r different n, we use 
to specify which n we are considering. 

Proposition 19. Let £,n G N + . Then we have 



rf i' rf 2 — d 1 + Q+rd,d 2 

In particular, let k = (™) ~~ <^i ~~ d 2 , then 

dimM dltd2 < p(d 2 , k). 

Proof. For any G , j)^ ltalan whose bi-degree is (di,d 2 ), we define 

D (n + t) = m Q); (1; o), ...,(£- 1, 0)} U (£><»> + (£, 0)), 

where L>( n ) + (£, 0) means translating the set .D^ by the vector (£,0). It is easy to verify 
that £K n +^ g D^f an has bi-degree (di + Q + n£,d 2 ). By Corollary US] we have proved the 
first assertion. 

For any DW g £) n of bi-degree (di,^), by taking sufficiently large £ and applying 
Proposition [12l we get 

A(D (n+£) ) = a M ■ A (^) (modulo lower degrees), 

where F M G 2)„+£ are special minimal staircase forms of bi-degree (di + (J + n£, ofo) and of 
partition type fi. This implies 



therefore 



dimM (n+ ^ <p(d 2 ,k), 



dim Mf\ < dim M {n+ %, < p(d 2 , k). 



□ 



5. The lower bound of dimM dljd2 
5.1. A homogeneous term order, leading terms and leading monomials. 
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Definition 20. (a) Let k G N + and denote by Q[p]k C Q[p] the vector space spanned by 
monomials p u = Y\p Vl for all sequences of positive integers v — {y\ < v 2 < .... < v m ) 
satisfying Vi = k. Equivalently, Q[p]k is the set of weighted homogeneous polynomials of 
weight k by assigning the weight of pi to be i, Wi G N + . 

(b) For any z/, p G Q[p]fc, denoted by v = {i>\ < v 2 < .... < v m ) and p = (pi < 
p 2 < .... < p n ), we define p v < p M if there is a positive integer j < min(m, n) such that 
Vi = pi for 1 < i < j — 1 and Uj < pj. This clearly defines a total order on Q[p]fc, VA; G N + . 

(c) For k G N + and a nonzero polynomial f = Yl a vPv e Q{p]k where a u G Q, we define 
the leading monomial of / to be the 

lm(/) := max{p u \a u ^ 0}, 

and define the leading term of / to be lt(/) := a u p u where p u = lm(/). For c G Q \ {0}, 
define lt(c) = 1 and lm(c) = c. □ 

Example 21. Let / = 2pip 2 p 7 — 5p 4 p 6 . Since p\p 2 p-j < PaPq, we have 

lm(/) = p 4 p 6 , lt(/) = -5p 4 p 6 . 

Lemma 22. (a) The total order of the monomials in Q[p]k defined in Definition IWi is 
preserved by multiplication: let p^, p^r, p v be monomials in Q[p]fc, then p M < p^i if and only 
if PtMpu < P^'pu- 

(b) Let p, v be monomials in Q[p]fc and p', v' be monomials in Q[p]/t' such that p M < p v 
and p^ < p v i . Then p^p^ < p v p v i . 

(c) for f G Q[p} k , g G Q[p] fe / (k, k' G N + ), we have 

lm(/#) = lm(/)lm(#), vr(fg) = lt(/)lt(#). 

Proof, (a) Denote p = (pi < p 2 <•••), p' = (p[ < p! 2 — ' ' ' )■ To show the "only if" part, 
suppose p < p! . It suffices to consider the case when we have a strict inequality pi < p[. 
Let p^ = p^p u and p^/ = p^>p u . Let t be the smallest integer satisfying vi > p\. Then 

f = (z/i, . . . ,^_i,pi, . . .), 

£' = (z/i,...,^_i,min(p / 1 ,z/f),...). 

Since pi < min(p' 1; vg), we have p^ < p% by definition and therefore p^p u < p^p u . On the 
other hand, the "if" part immediately follows from the "only if" part. 

(b) Applying (a) twice, we have p^p v > p^p u i > p^p u i. 

(c) It is an immediate consequence of (b). □ 
5.2. The theorems on the lower bound of dimM dl d2 . 

Theorem 23. Let n, k G N, k < n - 4. Let d u d 2 G N + ; d x + d 2 = g) - k, d 2 < d x . 
Then for each v G Hd 2 ,k, there exists a D u G D n , such that A(D U ) has bi-degree (di, d 2 ), and 
\M{ip(D v )) = p v . 
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Theorem 24. Letn,k e N, k < n-3. Letd 1 ,d 2 G N + ; d x + d 2 = {ty-k, d 2 < d x . Then for 
each v G n^^, there exists an alternating polynomial f v of bi-degree (di,d 2 ), either of the 
form A(D) or of the form A(D) — A(D') for some D, D' G D n , such that lm(^(/„)) = p v . 
Moreover, dim Md lt d 2 — p(d 2 , k), the partition number of k into at most d 2 parts. 



Remark 25. Theorem [241 gives a positive answer to Conjecture 8 in 1 1 j . Theorem [231 and 
Theorem [2H are proved using the same idea. In the proofs, we give explicit constructions 
for D v (in Theorem 123]) and /„ (in Theorem [241) . The constructions are non-canonical in 
the sense that there are choices to make, and it seems that no choice is more natural than 
others. 



Before we prove the above two theorems, we shall give an example to illustrate the idea 
of the construction. 

Example 26. We illustrate Theorem [23] by giving a construction of D v for n = 18, k = 14, 

(d±, d 2 ) = (84, 7), v = (1, 1, 1, 2, 2, 3, 4). First, we divide v into 3 sub-partitions V\ = (1, 1, 1), 
v 2 = (2,2), z> 3 = (3,4). For each sub-partition Pi, we construct D { G D' as follows: 



m m m 



D 9 



such that in the term order defined in §5.11 the leading monomials 

lm(<£> (Dj) ) = pp., fori = 1,2, 3. 

Now putting D 3 , D 2 , D\ together and adding appropriate extra points if necessary, we obtain 
D v as in the following graph. 



D a 



Do 



Di 



extra points 



D„ 







































2fc 




— -^H 








M 


M 


M 





It satisfies lm((/?(D i/ )) = lm(^(D x )) • lm(<^(D 2 )) • LM((p(D 3 )) = pp 1 pt> 2 pp 3 = p v . 



□ 



To generalize the above example, we need to separate a partition v into substrings 
V\,v 2 ,..., each of which contains at most 3 numbers. Every substring Vj corresponds to a 
Dj e D' satisfying LM.((p(Dj)) = pp.. The correspondence is specified in table f l5.2j) . Then by 
putting all Dj together and adding appropriate extra points if necessary, we obtain D e 25 
such that 



lm(>(D)) = JjLM((^(D i )) = Y[p . 



Pv 



5.3. Proof of the main theorem. The following crucial lemma provides an effective 
method to verify if a set of alternating polynomials is linearly independent by using ip. 
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Lemma 27. Fix (di,d 2 )- Let f G C[xi, yi, . . . , x n , y n ] e be a bi-homogeneous alternating 
polynomial of bi-degree (di,^). If tp(f) 7^ 0, i/ien / ^ modulo lower degrees. As a 
consequence, p> induces a well-defined linear map 

(p : M dl4 , 2 — ► C[pi,p 2 , ...]fc. 

Proof. Suppose </?(/) 7^ 0. By Proposition [T21 after replacing n by a sufficiently large integer 
if necessary, we can assume that / is linearly equivalent to a ^FpL modulo lower degrees, 
where are special minimal staircase forms. Since <p{f) 7^ 0, Proposition [12] guarantees 
dp 7^ for some \i. Using the fact that {A(F At )} M are linearly independent in Md ljC i 2 , we 
conclude that / ^ modulo lower degrees. □ 

The map <p is natural and useful in the study of Md u d 2 - Our main theorem (Theorem 
\2M implies that, for k := (™) — d\ — d 2 < ti — 3 (g^ < di), the map (p is injective and the 
image is spanned by {p u }uen d2 k - For more general k, we expect that the injectivity still 
holds. All the computations we did so far support this conjecture. 

Conjecture 28. The linear map <p is injective. 

In fact, we can show the following. 
Proposition 29. Conjecture \Uh implies Conjecture 12R 

Proof. Assume that Conjecture [13] is true. Suppose / G C[xi,yi, . . . , x n , y n ] e is a bi-homogeneous 
alternating polynomial of bi-degree (di,d 2 ) satisfying <p(f) = 0. 

Conjecture [13] implies that the elements of r £s c ^ talan with bi-degree (di,d 2 ) form a basis 
of M dl)rf2 , so we can express / as a linear combination ^Vcij A(Dj), D { G ^eaiaian. Define 
G S)^ /an as in the proof of Proposition [19] Then 

a, A(A')) = ^(E a * A ^)) = M = °' 

i i 

But 92 : ¥4^4 — > C[pi,p2, • • -]fc is injective (since k < (n + £) — 3 for sufficiently large f). 
So a, A(Z}-) = 0, which implies = Vz and therefore / = ^ i tjj A(Dj) = 0. □ 

Lemma 30. Le£ w>2eM. Suppose 

D = {P 1 ,...,P w+1 }e& w+1 , 

where Pi are all distinct and 

^1 = ^1 = 0, \Pi\=i-2, 3<i<w + l. 

Then the leading term 

LT(i P (D)) = (\P l \ y -\P 2 \ y ) Pw . 

In particular, the leading monomial 

LM((p(D)) = p w . 

Proof. Immediately follows from the definition of <p{D). □ 
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Lemma 31. Let v,w G N and 2 < v < w. Suppose 

D = {P 1 ,...,P w+2 }e& w+2 , 

where Pi are all distinct and 

{ 0, if i = 1,2; 
\Pi\ = < i-2, if3<i<w-v + 3; 

[ i-3, ifw-v + 4<i<w + 2. 

Then the leading term 

irr((fi(D)) = —(\Pl\y — \P2\y)(\Pw~v+3\y ~ I P-w-v+4 \y)PvPw ■ 

In particular, the leading monomial 

lm(<p(D)) = p v p w . 

Example 32. For v = 2, w = 3, D = {(-1, 1), (0, 0), (0, 1), (0, 2), (1, 1)}. 



A simple computation shows that 

tp(D) = -p 2 p3 + Pipi + p\p\ - 2pjp 3 + 2p\p 2 - p\, 
so the in:((p(D)) = — (1 — 0)(2 — l)p 2 P3 = — P2P3 as asserted in the above lemma. 

Proof of Lemma EE Suppose (p(D) = a^P^- First we show that a M 7^ implies p M < p v p v 
Suppose a M 7^ 0. There exist a G and integers {wj } such that the summand 

n 

sgn(cr) [[p^-PM 
i=l 

in (13.21) is not zero, and 

(5.1) ^=n^«^«'-"^«- 

i=l 

Because of condition (13. ip . we must have 

a{i) - 1 - at - hi > 0, Vz6[l,iy + 2], 

in particular, 

<j(w — f + 3) > w — f + 2, a(w — v + 4) > w — f + 2. 

Since a is a permutation, <r(w — u + 3) and o~(w — v + 4) are different from each other, hence 
at least one of them is greater than or equal to w — v + 3. Let abew- v + 3 or w — v + 4 
such that cr(tt) > w — v + 3. Since cr(it) < w + 2 and |P«|(= a u + b u ) = w — v + 1, we have 

1 < a(u) - 1 - \P U \ < v. 

By condition (13.11) . 

w( u) + ... + w[ u) = a{u) - 1 - a u - b u G [1, t> ], 
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Take j G N + , 1 < j < b u such that Wj ^ 0, then j9 w is a factor of p M by (15 .11) . Since 



Wj < v < w, Pn < p v p w . Therefore a M ^ implies p M < p^p^,. 



Now we show that ^ for p 



I. Assume the monomial p v p w appears in ( 15. ip . 



By the above argument, it is necessary that 

a(u) - 1 - \P U \ = v, 

which implies a{u) = w+2. Denote 5 = u— (w— v +3) e {0,1}. On the other hand, since cr(l) 
and <t(2) cannot be 1, we may assume <r(l + e) 7^ 1 for e G {0, 1}. Then a(l + e) — 1 — |Pi +e | = 
w, hence <r(l + e) = w + 1. For every positive integer % < w + 2 that i 7^ 1 + e, i 7^ «, we 
must have a{i) — 1 + |Pj|. So cr G 5„ must be one of the following. 



(7(1) 



1, if i = 2 - e; 

w + 1, if i = 1 + e; 

ife + 2<i<u;-t; + 2 + 5; 
u; + 2, if 2 = w — t> + 3 + <5; 
i - 2, ifw-w + 4 + 5<2<u; + 2, 



for (e, 5) = (0,0), (0,1), (1,0) or (1,1). By routine computation, 



e 


5 


coefficient of p v p w corresponding to a 










Pi 


y P-w—v+3 y 





1 


+ 


Pi 


y | P-w—v+A 


y 


1 





+ 


P2 


y Pw—v+3 


y 


1 


1 


— \p2 \y\Pw— v+A 


y 



Adding the above 4 coefficients gives 



<\Pl\y - Wv)(\P« —v+3 1 y \Pw— v+A\y 

)^o. 



□ 



Definition 33. To any sequence v = {v\ < is 2 < • • ■ < Vn\ of positive integers, we associate 
a sequence v = {z/j} of subsequences of u, each subsequence has the specified number of 
elements as follows. Denote by c the number of l's in v and m := n — c. 




1 or 2 



To be precise, 



r (1,1,1), i<i<r§i-i; 

(i i _ v ^i), « = r§i; 

c+3-3Tf 1 

(^c+2(i-rti)-i, ^ c +2(i-r§i)), r§l + 1 < * < m + \f 1 - 1; 
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Example 34. Uv = (9) then v = ((9)). 
If i/ = (1,1, 1,1) thenz>=((l,l,l),(l)). 
If v = (1, 1, 1, 1, 10) then z> = ((1, 1, 1), (1), (10)). 

If v = (1, 1, 1, 1, 1, 1, 1, 1, 3, 3, 5, 5) then ~v = ((1, 1, 1), (1, 1, 1), (1, 1), (3, 3), (5, 5)). 
If v = (1, 1, 1, 2, 2, 2, 3, 3, 7, 7) then z> = ((1, 1, 1), (2, 2), (2, 3), (3, 7), (7)). 

Proof of Theorem [HI The following table is the building block of our proof. In the table 
below, | /i | denotes the sum of all numbers in /i. 



(5.2) 





P^gID' 






(1,1,1) 


{Pl,p 2 ,p 3 }, IPI 




IP2 




p 3 | = 


3 


3 


(1,1) 


{-Pi, P2, P3, P4}, 


Pi 




P 2 


= 0, P 3 = P 4 = 2 


2 


4 


(1) 


{P u p 2 }, Pi = 


p 2 


= 


1 


2 


(v,w) 
2 < v < w 


{Pi,... 


, P w+2 } G S)^ +2 , sucn that 
r 0, if 1 < z < 2; 

z - 2, if 3 < % < w - v + 3; 

% - 3, ifw-w + 4<i<w + 2. 


t) + w 


w + 2 


H 

w > 2 


{Pi,... 

\P1\ = 1 


, P«;+l}, 

p 2 | = 0, 


such that 

Pi\ = i - 2 (3 < i < w + 1) 


w 


tw + 1 



We claim that, in the above table, the leading monomial lm(<^(£' /j )) = p M . Indeed, the 
case /i = (1,1,1) or (1) follows from Lemma [2] (v); the case fi = (1, 1) follows from Lemma 
[2](iv)(v); the case = (v,w) follows from Lemma I3H the case /i = (w) follows from Lemma 

nn 

Let v = {ui, . . . , P m } be defined as in Definition I3"B"1 The idea of the construction of D u 
is to take the union of translations of Ep x , . . . , E„ m together with some points in N x N that 
do not affect the value of <p. 

We consider 2 cases separately. 

CASE 1: u m ^ (1,1,1). 

Define translating vectors Ti, . . . , T m G N x N as follows. T m = (1,0), 

Ti = {l + #P 5i+1 + #P 5i+2 + • • • + #P 5m , 0), Vz G [1, m - 1]. 

Define 

n = 1 + + #P* 2 + • ■ ■ + #E Pm . 

Then n < (1 + \v x \ + |z> 2 | H V \v m \) + 3 = k + 4<n. Choose P, G N x N such that 

|Pj| = j — 1 for j G [ra + 1, n]. Define D G 3D' as follows, 

m n 

(5.3) ^ = {(0,0)} U U(/^-7;) U |J {P,}, 

i=l j=no+l 

where P^ + Tj denotes the set of points in N x N obtained by adding each point in E^ by 
the translating vector Tj. 
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Now we prove the following claim. 

Claim. Fix v G Ud 2 ,k- For any integer d 2 satisfying j^v < d 2 < (™) ~ k ~ {H^ u )i define 
d[ = (™) — k — d' 2 . Then we can make choices of and Pj in (15.31) , such that the bi-degree 
of D is (d[, d' 2 ), and the ^-coordinates of the points in D are non-negative, i.e. D G D. 

Proof of Claim. We give the exact lower bound and upper bound for the y-degree of D, and 
shows that any integers between the lower bound and upper bound can be the y-degree of 
some D. 



For the exact lower bound, we want to construct Pj and Ey i such that their y-degrees 
are as small as possible. We let Pj = (J — 1, 0) and E„. be as follows: 





E~ G 2)' 


y-degree of E^ 


(1,1,1) 


{(-2,2), (-1,1), (0,0)} 


3 


(1,1) 


{(-1,1), (0,0), (1,1), (0,2)} 


2 


(1) 


{(-1,1), (0,0)} 


1 


(v,w) 
2 < v < w 


{(0, 0), (1, 0), . . . , (w - 1, 0)} U {(-1, l),(w- v, 1)} 


2 


H 

w > 2 


{(-1,1), (0,0), (1,0),..., (10 -1,0)} 


1 



and denote the resulting D by D miny . Observe that the y-degree of E^. is equal to #z/j for 
all Vi in the table, so the y-degree of D miny is Y^iLii^^i) = (# Z7 )- 



For the exact upper bound, we need only to note that if D G D n can be constructed as 
(15. 3j) . then the transpose of D (i.e. swap the x and y coordinates of each point in D) can also 
be constructed as (15.31) for some choices of Pj and Ep.. In particular, the transpose of D miny , 
denoted by -D maxy , can be constructed as (15.31) . The y-degree of -D maxy is Q) — k — 
and is the maximal y-degree for all possible D G D n constructed as (15. 3p . 

Finally, by moving an appropriate point of D to the north-west direction, the y-degree 
increases by 1, so every integer between j^v and (™) — k — (#z/) is the y-degree of some D. 
This completes the proof of Claim. □ 

Now by assumption g?2 < d\ , d\ + g?2 = (2) — k, and (#^) < c?2 since v is a partition of k 
into no more than d^ parts. Therefore (#^) < ^2 < (2) — k — (#^) and by the above claim 
d2 is the y-degree of some D G 2) constructed as (15.31) . Take this D and denote it by D v . 
The bi-degree of D v is [d^d-i). Applying Lemma [2] (ii)(iii)(iv), 

m 

<p(D v ) = J[<P{E Pi ), 

i=l 

hence by Lemma [221 (c), 

m m 

hu(ip{D v )) = Y[lm((p(E J) = X\pu % = pv 

i=i i=i 



CASE 2: P m = (1,1,1). 
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In this case, {$v) = k = 3m. Choose D e D to satisfy: \Pj\ = j — 1 for 1 < j < n — 3m, 
|-Pn-3m+3j-2| = \Pn-3m+3j-i\ = \P n ~3m+3j\ = n = 3m + 3j - 3. By assumption, 1/ G U d2:k , 
so d 2 > k in this case. It is straightforward to verify that we can choose such a D to have 
bi-degree (d\, d 2 ). This completes the proof of Theorem [231 □ 

Proof of Theorem The proof is almost identical with the one of Theorem [231 We only 
need to modify the row p = (1, 1) in the table (15. 2p . Instead of using E^u e D' (which 
contains 4 points), we use two elements EL^ and E'L^ in D', each of which contains 3 
points. 

-^(1,1) = {( -a _ 1) a + 1); {- a i a), (a + 1, «)}, 
£(1,1) = - 1, a + 1), (-a, a), (a, a + 1)}. 

A simple computation shows 

¥>(-E[i,i)) = P2, <f(E" ltl) ) = -p\ + p 2 , 

so 

<p(E[ ltl) ) - p(E( 1}1) ) = p\. 

Here we need to be cautious that the bi-degree of EL ^ and E'L X s are not the same. This will 
not bring any problem, since we can move points in other E^ to adjust the total bi-degree. 
Eventually, supposing thata t is the integer that = (1, 1), we can construct D' u , D" e 2) 
both of bi-degree (di, d 2 ) such that 

<p(D'„) = cp(E{ ltl) ) 11^), y{D'l) = <p(E'{ LA) )Y[<p(E i>i ). 

Then / := A(D' V ) - A(D'l) satisfies lm(^(/)) = p u . 

Now for each v e 11^^, we can construct f u such that lm(<^(/)) = p u . If we write down 
the coefficient matrix for ip(f u ) with basis {p^}^u k arranged in decreasing order, we obtain 
a row echelon form with rank p(d 2 , k). So dim Md lt d 2 — p(^2 ; k) by Lemma [2T1 Combining 
the upper bound obtained in Proposition [T9| we conclude that dim Md lt d 2 = p(d 2 , k). □ 

6. The condition for the equality dimM dljd2 =p(d 2 ,k) to hold 

In Proposition [191 we showed the inequality dim Md lt d 2 < p(d2,k), then in Theorem [241 
we showed that "=" holds for k < n — 3. In this section, we show that the condition k < n — 3 
is the best we can hope, in the sense of the following theorem. 

Theorem 35. Assume d 2 < d\. Then dim Md 1 ^ 2 — p{d2,k), and the equality holds if and 
only if "k <n-3 ", or "k = n - 2 and d 2 = 1 or u d 2 = 

Proof. The inequality is proved in Proposition [191 Then we verify the equality dimM^^^ — 
p(d 2 , k) in the specified 3 cases. The case d 2 = is trivial since by definition p(0, k) = for 
k > 1 and p(0, 0) = 1, we can check the equality directly. In the case k = n — 2 and d 2 = 1, 
dimM dl , d2 = 1 because A({(0, 0), (0, 1), (1, 0), (2, 0), . . . , (n - 2, 0)}) forms a basis for M dl42 . 
The case k < n — 3 is proved in Theorem [2H 
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Now assume d 2 > 2. We use the notation M^ d to specify which n we are considering. 
By Proposition [19j h suffices to show that dim M^ ds < dimM^ d2 for k = n - 2. 

Using the condition d\ > d 2 , it is easy to check that (di — n + 3) > 0. So both (di — n + 3) 
and (d 2 — 2) are non- negative integers and (d± — n + 3) + (d 2 — 2) = Q) — ^~ n+1 = (™2 2 )- We 

know that dimM^I^^ = 1. Let D^-^ E T) c n ^ 2 alan be of bi-degree {d l -n + 3, d 2 - 2). 
Define 

L^+D = {(0, 0), (1, 0), (0, 2)} U (D^ + (2, 0)) . 

Then D (n+1) e ®%ff an is of bi-degree (d l +n, d 2 ). On the other hand, every e f £)^ atalan 
of bi-degree (di,d 2 ) determines an element 

{(0,0)}U (£><"> + (1,0)) 

in f Q^° lan of bi-degree (c?i + n, d 2 ), which is distinct from D( n+1 \ Therefore 

dimMa 2 <dimM^ d2 . 

□ 

Remark 36. For k — Q) — d\ — d 2 = 0, we know dimM^ d = p(d 2 ,k) = 1. We give a 
straightforward construction of the element D e ^Catalan Q f D i_cl e g ree (c?i, cfe) as follows. Let 
tt= L(2n + 1 - V4n 2 - 4n - 8^ + 9 ) /2J , i = nu - u(u + l)/2 - di, define 

(xi, . . . , x n ) = (u — l,u— — . . . ,u,u — l,u — 2,. ..,1,0), 



n—u—i 



Vi = #{j I i < j, a?j- - ar< e {0, 1}}, i = 1, . . . , n. 
Then D = {(x u y x ), (x 2 , y 2 ), ■ ■ ■ , (ac„, &/»)}. □ 
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7. Appendix 

7.1. Table of the q,t Catalan number for n = 7. The number located at the (i,j)-th 
coordinate is equal to the coefficient of q l P in CV(g, t). The left at the bottom is the (0, 0)-th 
position. 

1 

1 

1 1 

111 

12 11 

12 2 11 

1 3 3 2 1 1 

2 4 3 2 1 1 

2 4 5 3 2 1 1 

1 4 5 5 3 2 1 1 

1 3 6 6 5 3 2 1 1 



257653211 

1468653211 

2578653211 

13688653211 

13678653211 

13567653211 

12456553211 

1234443211 

112232211 

111111 



000000000000000000001 
Tables for n < 7 can be found at F. Bergeron's website 
\protect\vrule widthOpt\protect\href {http : / /bergeron . math . uqam . ca/ n_f act_Conj ecture/qt_c 

7.2. Macaulay 2 code for computing ip. For the convenience of the reader, we provide a 
Macaulay 2 code [11] for computing ip (function "phi" in the code) defined in §2.1, together 
with an example of computation 

^({(-1,1), (0,0), (0,1), (0,2), (1,1)}). 

il : phi=(D)->( 

local R,n,k, sgn, s , total , t ,bi , sumrho ,prod; 
n=#D; 
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R=ZZ[r_0. .r_n] ; 

k=n*(n-l)//2; for i from 1 to n do 

k=k-D#(i-l)#0-D#(i-l)#l; 
total=0; 

scan (permutations (n) ,sigma->( 
sgn=l; 

for i from to n do 

for j from i+1 to #sigma-l do 
(if sigma#i>sigma#j then sgn=sgn* (-1) ) ; 
t=l; 

for i from 1 to n do 
( 

ai=D#(i-l)#0;bi=D#(i-l)#l; 
s=(sigma#(i-l)+l)-l-ai-bi ; 
if (s<0) then (t=0;break) else 
if (bi==0) and (s>0) then (t=0; break) else 
if (bi==0) and (s==0) then (t=t*l) else 
if (bi==l) then (t=t*r_s) else 
(sumrho=0; 

scan(subsets (s+bi-1 ,bi-l) ,su->( 
prod=r_ (su#0) ; 
for j from 2 to bi-1 do 

prod=prod*r_ (su# ( j -1) -su# ( j -2) -1) ; 
prod=prod*r_(s+bi-2-su#(bi-2) ) ; 
sumrho=sumrho+prod; ) 

); 

t=t*sumrho ; 

); 

); 

total=total+sgn*t ; 
)); — end of scan of sigma. 
return sub( (-1) ~k*total , {r_0=>l}) ; 

) 

ol = phi 

ol : FunctionClosure 

i2 : phi({(-l,l),(0,0),(0,l),(0,2),(l,l)}) 

5 3 2 2 

o2=-r +2rr +rr -2rr -rr +rr 
1 12 12 13 23 14 



o2 : R 
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